\section{Introduction}
AI systems are capable of playing specific video games, such as Super Mario World \cite{Karakovskiy2012}  and Starcraft \cite{Young2012}, with comparable skill to expert human players. However, all such AI systems rely on a human to somehow perform the challenging and tedious task of specifying the game rules, objectives and entities.

For example, state-of-the-art AI systems for playing Mario and Starcraft can play these games effectively, even when faced with challenging and complex game states. However, these systems rely heavily on hand-crafted heuristics and search algorithms that are specific to the game they target, and are not readily generalizable to other games.

In contrast, systems for General Game Playing (GGP) \cite{Genesereth2005}, such as CadiaPlayer \cite{Bjornsson2009}, can play novel games for which they were not specifically designed. However, GGP systems rely on a human to provide a complete formal specification of the game rules, objectives, and entities in a logical programming language similar to Prolog. Arriving at such a formal specification is very tedious even for the simplest games. This limitation significantly constrains the applicability of GGP systems.

\begin{figure}
\centering
\includegraphics[width=0.47\textwidth]{figures/teaser.png}
\caption{Our system successfully learns to play the games shown above: \textsc{Eat-The-Fruit} (top-left), \textsc{Pong} (top-middle), \textsc{Dance-Dance-Revolution} (top-right), \textsc{Frogger} (bottom-left), \textsc{Snake} (bottom-middle), \textsc{Dodge-The-Missile} (bottom-right).}
\vspace{-1pc}
\end{figure}

Very recently, Bellemare \etal \cite{bellemare12arcade} released the Arcade Learning Environment for evaluating the performance of AI agents on a large set of Atari 2600 games. In this work, Bellemare \etal evaluate a variety of feature transforms that generalize across 2D games, as well as evaluating the SARSA($\lambda$) algorithm for online model-free reinforcement learning in this setting. Bellemare \etal demonstrate that the SARSA($\lambda$) achieves reasonable performance on a large variety of games.

In this project, we aim for comparable performance and generality to that recently demonstrated by Bellemare \etal \cite{bellemare12arcade}. Indeed, our technical approach is directly inspired by their work. To our knowledge, the Arcade Learning Environment is the only system to implement AI agents that learn to play such a wide variety of non-trivial games. Therefore, it is worth emphasizing that the games we consider in this project are at the approximate boundary of what general AI agents are capable of learning. This is true despite the apparent simplicity of our games, even compared to classic 2D games like Super Mario World.
